Lower Bounds on the Generalization Error of Nonlinear Learning Models

We study in this paper lower bounds for the generalization error of models derived from multi-layer neural networks, regime where size layers is commensurate with number samples training data. derive explicit general biased estimators, cases two-layered networks. For linear activation function, bound asymptotically tight. In nonlinear case, we provide a comparison our an empirical stochastic gradient descent algorithm. addition, unbiased which show that latter have unacceptable performance truly The analysis uses elements theory large random matrices.